07. Metric - Click Through Rate

Let's recap the steps we took to analyze the results of this A/B test.

1. We computed the observed difference between the metric, click through rate, for the control and experiment group.
2. We simulated the sampling distribution for the difference in proportions (or difference in click through rates).
3. We used this sampling distribution to simulate the distribution under the null hypothesis, by creating a random normal distribution centered at 0 with the same spread and size.
4. We computed the p-value by finding the proportion of values in the null distribution that were greater than our observed difference.
5. We used this p-value to determine the statistical significance of our observed difference.